3D image capture device

ABSTRACT

This 3D image capture device includes: a light-transmitting section  1  including first and second light-transmitting areas  1 L and  1 R; an image sensor  2   a  arranged to receive the light transmitted through the light-transmitting section  1 ; an imaging section  3  that produces an image on an imaging area of the image sensor  2   a ; and an image capturing driving section that drives the image sensor so as to perform image capturing sessions at least twice in a row and drives the light-transmitting section so that the first and second light-transmitting areas change their positions every image capturing session. The first light-transmitting area  1 L transmits light falling within a first wavelength range included in a color red wavelength range and light falling within a second wavelength range included in a color green wavelength range. The second light-transmitting area  1 R transmits light falling within a third wavelength range, included in the color green wavelength range and shorter than the second wavelength range, and light falling within a fourth wavelength range included in a color blue wavelength range.

TECHNICAL FIELD

The present application relates to a single-lens 3D image capturingtechnology for generating a parallax image using a single optical systemand a single image sensor.

BACKGROUND ART

Recently, the performance and functionality of digital cameras anddigital movie cameras that use some solid-state image sensor such as aCCD and a CMOS (which will be sometimes simply referred to herein as an“image sensor”) have been enhanced to an astonishing degree. Inparticular, the size of a pixel structure for use in a solid-state imagesensor has been further reduced these days thanks to rapid developmentof semiconductor device processing technologies, thus getting an evengreater number of pixels and drivers integrated together in asolid-state image sensor. As a result, the resolution of an image sensorhas lately increased rapidly from around one million pixels to tenmillion or more pixels in a matter of few years. On top of that, thequality of an image captured has also been improved significantly aswell.

As for display devices, on the other hand, LCD and plasma displays witha reduced depth now provide high-resolution and high-contrast images,thus realizing high performance without taking up too much space. Andsuch video quality improvement trends are now spreading from 2D imagesto 3D images. In fact, 3D display devices that achieve high imagequality although they require the viewer to wear a pair of polarizationglasses have been developed just recently.

As for the 3D image capturing technology, a typical 3D image capturedevice with a simple arrangement uses an image capturing system with twocameras to capture a right-eye image and a left-eye image. According tothe so-called “two-lens image capturing” technique, however, two camerasneed to be used, thus increasing not only the overall size of the imagecapture device but also the manufacturing cost as well. To overcome sucha problem, methods for capturing multiple images with parallax (whichwill be sometimes referred to herein as a “multi-viewpoint image”) byusing a single camera have been researched and developed. Such a methodis called a “single-lens image capturing method”.

For example, Patent Document No. 1 discloses a scheme that uses twopolarizers, of which the transmission axes cross each other at rightangles, and a rotating polarization filter. FIG. 11 is a schematicrepresentation illustrating an arrangement for an image capturing systemthat adopts such a scheme. This imaging area includes a0-degree-polarization polarizer 11, a 90-degree-polarization polarizer12, a reflective mirror 13, a half mirror 14, a circular polarizationfilter 15, a driver 16 that rotates the circular polarization filter 15,an optical lens 3, and an image capture device 29 which captures theimage that has been produced by the optical lens 3. In this arrangement,the half mirror 14 reflects the light that has been transmitted throughthe polarizer 11 and then reflected from the reflective mirror 13 buttransmits the light that has been transmitted through the polarizer 12.With such an arrangement, the light beams that have been transmittedthrough the two polarizers 11 and 12, which are arranged at twodifferent positions, pass through the half mirror 14, the circularpolarization filter 15 and the optical lens 3 and then enter the imagecapture device 29, where an image is captured. The image capturingprinciple of this scheme is that two images with parallax are capturedby rotating the circular polarization filter 15 so that the light beamsthat have been incident on the two polarizers 11 and 12 are imaged atmutually different times.

According to such a scheme, however, images at mutually differentpositions are captured time-sequentially by rotating the circularpolarization filter 15, and therefore, two images with parallax cannotbe captured at the same time, which is a problem. On top of that, sinceall of the incoming light passes through the polarizers 11, 12 and thepolarization filter 15, the quantity of the light received eventually bythe image capture device 29 decreases by as much as 50%, which isnon-negligible, either.

To overcome these problems, Patent Document No. 2 discloses a scheme forcapturing two images with parallax at the same time without using suchmechanical driving. An image capture device that adopts such a schemegets the two incoming light beams, which have come from two differentdirections, condensed by a reflective mirror, and then received by animage sensor in which two different kinds of polarization filters arearranged alternately, thereby capturing two images with parallax withoutusing mechanical driving.

FIG. 12 is a schematic representation illustrating an arrangement for animage capturing system that adopts such a scheme. This image capturingsystem includes two polarizers 11 and 12, of which the transmission axesare arranged to cross each other at right angles, reflective mirrors 13,an optical lens 3, and an image sensor 2. On its imaging area, the imagesensor 2 has a number of pixels 10 and polarization filters 17 and 18,each of which is provided one to one for an associated one of thepixels. Those polarization filters 17 and 18 are arranged alternatelyover all of those pixels. In this case, the transmission axis directionsof the polarization filters 17 and 18 agree with those of the polarizers11 and 12, respectively.

With such an arrangement, the incoming light beams are transmittedthrough the polarizers 11 and 12, reflected from the reflective mirrors13, passed through the optical lens 3 and then incident on the imagingarea of the image sensor 1. Those light beams to be transmitted throughthe polarizers 11 and 12, respectively, and then incident on the imagesensor 1 are transmitted through the polarization filters 17 and 18 andthen photoelectrically converted by the pixels that are located rightunder those polarization filters 17 and 18. If the images to be producedby those light beams that have been transmitted through the polarizers11 and 12 and then incident on the image sensor 1 are called a“right-eye image” and a “left-eye image”, respectively, then theright-eye image and the left-eye images are generated by a group ofpixels that face the polarization filters 17 and a group of pixels thatface the polarization filter 18, respectively.

As can be seen, according to the scheme disclosed in Patent Document No.2, two kinds of polarization filters, of which the transmission axes arearranged so as to cross each other at right angles, are arrangedalternately over the pixels of the image sensor, instead of using thecircular polarization filter disclosed in Patent Document No. 1. As aresult, although the resolution decreases to a half compared to themethod of Patent Document No. 1, a right-eye image and a left-eye imagewith parallax can be obtained at the same time by using a single imagesensor. According to such a technique, however, the incoming light hasits quantity decreased considerably when being transmitted through thepolarizers and the polarization filters, and therefore, the quantity ofthe light received by the image sensor decreases as significantly as inPatent Document No. 1.

To cope with such a problem of the decreased quantity of light received,Patent Document No. 3 discloses a technique for obtaining two imageswith parallax and a normal image with a single image sensor. Accordingto such a technique, those two images with parallax and the normal imagecan be obtained by a single image sensor by changing mechanically somecomponents that have been used to capture two images with parallax withalternative components for use to capture a normal image, and viceversa. When two images with parallax are going to be obtained, twopolarization filters are arranged on the optical path as disclosed inPatent Document No. 2. On the other hand, when a normal image is goingto be obtained, those polarization filters are mechanically removed fromthe optical path. By introducing such a mechanism, those images withparallax and a normal image that uses the incoming light highlyefficiently can be obtained.

Although a polarizer or a polarization filter is used according to thetechniques disclosed in Patent Document Nos. 1 to 3, color filters mayalso be used according to another approach. For example, Patent DocumentNo. 4 discloses a technique for obtaining two images with parallax atthe same time using color filters. FIG. 13 schematically illustrates animage capturing system as disclosed in Patent Document No. 4. The imagecapturing system includes a lens 3, a lens diaphragm 19, a light beamconfining plate 20 with two color filters 20 a and 20 b that havemutually different transmission wavelength ranges, and a photosensitivefilm 21. In this case, the color filters 20 a and 20 b may be filtersthat transmit red- and blue-based light rays, respectively.

In such an arrangement, the incoming light passes through the lens 3,the lens diaphragm 19 and the light beam confining plate 20 and producesan image on the photosensitive film. In the meantime, only red- andblue-based light rays are respectively transmitted through the two colorfilters 20 a and 20 b of the light beam confining plate 20. As a result,a magenta-based color image is produced on the photosensitive film bythe light rays that have been transmitted through the two color filters.In this case, since the color filters 20 a and 20 b are arranged atmutually different positions, the image produced on the photosensitivefilm comes to have parallax. Thus, if a photograph is developed with thephotosensitive film and viewed with a pair of glasses, in which red andblue films are attached to its right- and left-eye lenses, the viewercan view an image with depth. In this manner, according to the techniquedisclosed in Patent Document No. 4, a multi-viewpoint image can beproduced using the two color filters.

According to the technique disclosed in Patent Document No. 4, the lightrays are imaged on the photosensitive film, thereby producing imageswith parallax there. Meanwhile, Patent Document No. 5 discloses atechnique for producing images with parallax by transforming incominglight into electrical signals. FIG. 14 schematically illustrates a lightbeam confining plate according to Patent Document No. 5. According tosuch a technique, a light beam confining plate 22, which has a red raytransmitting R area 22R, a green ray transmitting G area 22G and a blueray transmitting B area 22B, is arranged on a plane that intersects withthe optical axis of the imaging optical system at right angles. And bygetting the light rays that have been transmitted through those areasreceived by a color image sensor that has red-, green- andblue-ray-receiving R, G and B pixels, an image is generated based on thelight rays that have been transmitted through those areas.

Patent Document No. 6 also discloses a technique for obtaining imageswith parallax using a similar configuration to the one used in PatentDocument No. 5. FIG. 15 schematically illustrates a light beam confiningplate as disclosed in Patent Document No. 6. According to thattechnique, by making the incoming light pass through R, G and B areas23R, 23G and 23B of the light beam confining plate 23, images withparallax can also be produced.

Patent Document No. 7 also discloses a technique for generating multipleimages with parallax using a pair of filters with mutually differentcolors, which are arranged symmetrically to each other with respect toan optical axis. By using red and blue filters as the pair of filters,an R pixel that senses a red ray observes the light that has beentransmitted through the red filter, while a B pixel that senses a blueray observes the light that has been transmitted through the bluefilter. Since the red and blue filters are arranged at two differentpositions, the light received by the R pixel and the light received bythe B pixel have come from mutually different directions. Consequently,the image observed by the R pixel and the image observed by the B pixelare ones viewed from two different viewpoints. And by definingcorresponding points between those images on a pixel-by-pixel basis, themagnitude of parallax can be calculated.

And based on the magnitude of parallax calculated and information aboutthe focal length of the camera, the distance from the camera to thesubject can be obtained. Patent Document No. 8 discloses a technique forobtaining information about a subject distance based on two images thathave been generated using either a diaphragm to which two color filterswith mutually different aperture sizes are attached or a diaphragm towhich two color filters in two different colors are attachedhorizontally symmetrically with respect to the optical axis. Accordingto such a technique, if light rays that have been transmitted throughred and blue color filters with mutually different aperture sizes areobserved, the degrees of blur observed vary from one color to another.That is why the degrees of blur of the two images that are associatedwith the red and blue color filters vary according to the subjectdistance. By defining corresponding points with respect to those imagesand comparing their degrees of blur to each other, information about thedistance from the camera to the subject can be obtained. On the otherhand, if light rays that have been transmitted through two color filtersin two different colors that are attached horizontally symmetricallywith respect to the optical axis are observed, the direction from whichthe light observed has come changes from one color to another. As aresult, two images that are associated with the red and blue colorfilters become images with parallax. And by defining correspondingpoints with respect to those images and calculating the distance betweenthose corresponding points, information about the distance from thecamera to the subject can be obtained.

According to the techniques disclosed in Patent Documents Nos. 4 to 8mentioned above, images with parallax can be produced by arranging RGBcolor filters on a light beam confining plate. However, since a lightbeam confining plate is used, the percentage of the incoming light thatcan be used decreases significantly. In addition, to increase the effectof parallax, those RGB color filters should be arranged at distantpositions and should have decreased areas. In that case, however, thepercentage of the incoming light that can be used further decreases.

Unlike these techniques, Patent Document No. 9 discloses a technique forobtaining multiple images with parallax and a normal image that is freefrom the light quantity problem by using a diaphragm in which RGB colorfilters are arranged. According to that technique, when the diaphragm isclosed, only the light rays that have been transmitted through the RGBcolor filters are received. On the other hand, when the diaphragm isopened, the RGB color filter areas are outside of the optical path, andtherefore, the incoming light can be received entirely. Consequently,images with parallax can be obtained when the diaphragm is closed and anormal image that uses the incoming light highly efficiently can beobtained when the diaphragm is opened.

CITATION LIST Patent Literature

-   Patent Document No. 1: Japanese Laid-Open Patent Publication No.    62-291292-   Patent Document No. 2: Japanese Laid-Open Patent Publication No.    62-217790-   Patent Document No. 3: Japanese Laid-Open Patent Publication No.    2001-016611-   Patent Document No. 4: Japanese Laid-Open Patent Publication No.    2-171737-   Patent Document No. 5: Japanese Laid-Open Patent Publication No.    2002-344999-   Patent Document No. 6: Japanese Laid-Open Patent Publication No.    2009-276294-   Patent Document No. 7: Japanese Laid-Open Patent Publication No.    2010-38788-   Patent Document No. 8: Japanese Laid-Open Patent Publication No.    2010-79298-   Patent Document No. 9: Japanese Laid-Open Patent Publication No.    2003-134533

Non-Patent Literature

-   Non-Patent Document No. 1: Yuta MORIUE, Takeshi TAKAKI, and Idaku    ISHII, A Real-time Monocular Stereo System Using a Viewpoint    Switching Iris, Transactions of the 27^(th) Annual Conference of the    Robotics Society of Japan, 3R2-06, 2009.

SUMMARY OF INVENTION Technical Problem

Even if such a traditional technique that uses either polarizers orcolor filters is adopted, multi-viewpoint images can be certainlygenerated, but the quantity of the light that eventually enters theimage sensor decreases significantly as the incoming light needs to passthrough the polarizers or color filters. A normal image that uses theincoming light highly efficiently can be obtained by using a mechanismthat removes either the polarizing areas or color filters from theoptical path in order to make a sufficient quantity of light incident.However, even if such an arrangement is adopted, the percentage of thelight that can be used to generate the multi-viewpoint images themselvesis still low and the same old problem persists.

Thus, in order to overcome such problems while taking into account thefact that color filters can be made at a lower cost than polarizers, anembodiment of the present invention provides an image capturingtechnology, by which multi-viewpoint images can be obtained with theincoming light used highly efficiently by using color filters.

Solution to Problem

To overcome these problems, a 3D image capture device according to anaspect of the present invention includes: a light-transmitting sectionincluding first and second light-transmitting areas; an image sensorthat is arranged to receive the light that has been transmitted throughthe light-transmitting section; an imaging section that produces animage on an imaging area of the image sensor; and an image capturingdriving section that drives the image sensor and the light-transmittingsection. The first light-transmitting area has a property to transmitlight falling within a first wavelength range that is included in acolor blue wavelength range and light falling within a second wavelengthrange that is included in a color green wavelength range. The secondlight-transmitting area has a property to transmit light falling withina third wavelength range, which is included in the color greenwavelength range and of which the wavelength is longer than the secondwavelength range, and light falling within a fourth wavelength rangethat is included in a color red wavelength range. The image capturingdriving section drives the image sensor so as to perform image capturingsessions at least twice in a row and drives the light-transmittingsection so that the first and second light-transmitting areas changetheir positions with each other every time an image capturing session iscarried out.

This general and particular embodiment can be implemented as a system, amethod, a computer program or a combination thereof.

Advantageous Effects of Invention

According to an embodiment of the present invention, multi-viewpointimages can be generated with the incoming light used more efficientlythan in the related art.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 A block diagram illustrating a configuration for an image capturedevice as a first embodiment.

FIG. 2 Schematically illustrates a general arrangement of alight-transmitting plate, an optical lens and an image sensor in thefirst embodiment.

FIG. 3 A front view of the light-transmitting plate according to thefirst embodiment.

FIG. 4 A graph showing the spectral transmittance characteristic of thelight-transmitting plate according to the first embodiment.

FIG. 5 Illustrates the basic color arrangement of an image sensoraccording to the first embodiment.

FIG. 6 A graph showing the spectral transmittance characteristics ofrespective color elements of the image sensor according to the firstembodiment.

FIG. 7 Illustrates how the light-transmitting plate is rotated accordingto the first embodiment.

FIG. 8A A graph showing another exemplary spectral transmittancecharacteristic of a light-transmitting plate as a modified example ofthe first embodiment.

FIG. 8B A front view of the light-transmitting plate as a modifiedexample of the first embodiment.

FIG. 8C Illustrates the basic color arrangement according to a modifiedexample of the first embodiment.

FIG. 8D A graph showing the spectral transmittance characteristics ofrespective color elements of the image sensor according to a modifiedexample of the first embodiment.

FIG. 9 Illustrates configurations for a light-transmitting plateaccording to modified examples of the first embodiment.

FIG. 10 A front view of a light-transmitting plate as a secondembodiment.

FIG. 11 Illustrates the arrangement of an image capturing systemaccording to Patent Document No. 1.

FIG. 12 Illustrates the arrangement of an image capturing systemaccording to Patent Document No. 2.

FIG. 13 Illustrates the arrangement of an image capturing systemaccording to Patent Document No. 4.

FIG. 14 Illustrates the appearance of a light beam confining plateaccording to Patent Document No. 5.

FIG. 15 Illustrates the appearance of a light beam confining plateaccording to Patent Document No. 6.

DESCRIPTION OF EMBODIMENTS

(1) To overcome the problems described above, a 3D image capture deviceaccording to an aspect of the present invention includes: alight-transmitting section including first and second light-transmittingareas, the first light-transmitting area having a property to transmitlight falling within a first wavelength range that is included in acolor blue wavelength range and light falling within a second wavelengthrange that is included in a color green wavelength range, the secondlight-transmitting area having a property to transmit light fallingwithin a third wavelength range, which is included in the color greenwavelength range and of which the wavelength is longer than the secondwavelength range, and light falling within a fourth wavelength rangethat is included in a color red wavelength range; an image sensor thatis arranged to receive the light that has been transmitted through thelight-transmitting section; an imaging section that produces an image onan imaging area of the image sensor; and an image capturing drivingsection that drives the image sensor so as to perform image capturingsessions at least twice in a row and that drives the light-transmittingsection so that the first and second light-transmitting areas changetheir positions with each other every time an image capturing session iscarried out.

(2) In one embodiment, the 3D image capture device of (1) furtherincludes an image processing section that generates multi-viewpointimages based on a pixel signal supplied from the image sensor.

(3) In one embodiment of the 3D image capture device of (1) or (2), theimage sensor has a plurality of pixel blocks that are arrangedtwo-dimensionally on the imaging area, and each of the plurality ofpixel blocks includes first, second, third and fourth pixels that mainlysense light rays falling within the first, second, third and fourthwavelength ranges, respectively.

(4) In one embodiment of the 3D image capture device of one of (1) to(3), the first and second light-transmitting areas are configured sothat the sum of a function representing a spectral transmittancecharacteristic of the first light-transmitting area and a functionrepresenting a spectral transmittance characteristic of the secondlight-transmitting area does not have wavelength dependence.

(5) In one embodiment of the 3D image capture device of one of (1) to(4), the light-transmitting section further includes a transparent area.

(6) In one embodiment of the 3D image capture device of one of (1) to(5), the second wavelength range is from 500 nm to 550 nm and the thirdwavelength range is from 550 nm to 600 nm.

(7) In one embodiment of the 3D image capture device of one of (1) to(6), the image sensor has a Bayer type pixel arrangement.

(8) In one embodiment of the 3D image capture device of one of (1) to(7), the first and second light-transmitting areas are arrangedsymmetrically with respect to the center of the light-transmittingsection, and the image capturing driving section rotates thelight-transmitting section 180 degrees on the center of thelight-transmitting section as the axis of rotation, thereby changing thepositions of the first and second light-transmitting areas with eachother every image capturing session.

(9) An image processor according to an aspect of the present inventiongenerates multi-viewpoint images based on a signal supplied from a 3Dimage capture device. The device includes: a light-transmitting sectionincluding first and second light-transmitting areas, the firstlight-transmitting area having a property to transmit light fallingwithin a first wavelength range that is included in a color bluewavelength range and light falling within a second wavelength range thatis included in a color green wavelength range, the secondlight-transmitting area having a property to transmit light fallingwithin a third wavelength range, which is included in the color greenwavelength range and of which the wavelength is longer than the secondwavelength range, and light falling within a fourth wavelength rangethat is included in a color red wavelength range; an image sensor thatis arranged to receive the light that has been transmitted through thelight-transmitting section; an imaging section that produces an image onan imaging area of the image sensor; and an image capturing drivingsection that drives the image sensor so as to perform image capturingsessions at least twice in a row and that drives the light-transmittingsection so that the first and second light-transmitting areas changetheir positions with each other every time an image capturing session iscarried out.

Hereinafter, embodiments of the present invention will be described infurther detail with reference to the accompanying drawings. In thefollowing description, any element shown in multiple drawings and havingthe same or similar function will be identified by the same referencenumeral. It should be noted that a signal or information representing animage will be sometimes referred to herein as just an “image”.

Embodiment 1

FIG. 1 is a block diagram illustrating an overall configuration for animage capture device as a first embodiment of the present invention. Theimage capture device of this embodiment is a digital electronic cameraand includes an image capturing section 100 and a signal processingsection 200 that generates a signal representing an image (i.e., animage signal) based on the signal generated by the image capturingsection 100.

The image capturing section 100 includes a color solid-state imagesensor 2 a (which will be simply referred to herein as an “imagesensor”) with a number of photosensitive cells that are arranged on itsimaging area, a light transmitting plate (light-transmitting section) 1with two light-transmitting areas, of which the transmittances havedifferent wavelength dependences (i.e., different spectral transmittancecharacteristics), an optical lens 3 for producing an image on theimaging area of the image sensor 2 a, and an infrared cut filter 4. Theimage capturing section 100 further includes a signal generating andreceiving section 5, which not only generates a fundamental signal todrive the image sensor 2 a but also receives the output signal of theimage sensor 2 a and sends it to the signal processing section 200, asensor driving section 6 for driving the image sensor 2 a in accordancewith the fundamental signal generated by the signal generating andreceiving section 5, and a rotating and driving section 9 which rotatesthe light-transmitting plate 1. In this embodiment, the signalgenerating and receiving section 5, the sensor driving section 6 and therotating and driving section 9 functions, in combination, as an imagecapturing driving section according to the present invention.

The image sensor 2 a is typically a CCD or CMOS sensor, which may befabricated by known semiconductor device processing technologies. Thesignal generating and receiving section 5 and the sensor driving section6 may be implemented as an LSI such as a CCD driver. The rotating anddriving section 9 has a motor that rotates the light-transmitting plate1 and can rotate, along with the sensor driving section 6, thelight-transmitting plate 1 on its center as the axis of 27 rotation.

The signal processing section 200 includes an image processing section 7which processes the output signal of the image capturing section 100 togenerate multi-viewpoint images, a memory 30 which stores various kindsof data for use to generate the image signal, and an interface (I/F)section 8 which sends out the image signal thus generated to an externaldevice. The image processing section 7 may be a combination of ahardware component such as a known digital signal processor (DSP) and asoftware program for use to perform image processing involving the imagesignal generation. The memory 30 may be a DRAM, for example. And thememory 30 not only stores the signal supplied from the image capturingsection 100 but also temporarily retains the image data that has beengenerated by the image processing section 7 or compressed image data.These image data are then output either a storage medium or a displaysection (neither is shown) by way of the interface section 8.

The image capture device of this embodiment actually further includes anelectronic shutter, a viewfinder, a power supply (or battery), aflashlight and other known components. However, the description thereofwill be omitted herein because none of them are essential componentsthat would make it difficult to understand how this embodiment worksunless they were described in detail. Also, this configuration is onlyan example. Thus, in this embodiment, additional components other thanthe light-transmitting plate 1, the image sensor 2 a and the imageprocessing section 7 may be implemented as an appropriate combination ofknown elements.

Next, the configuration of the image capturing section 100 will bedescribed in further detail. In the following description, the x and ycoordinates shown in the drawings will be used.

FIG. 2 schematically illustrates the relative arrangement of thelight-transmitting plate 1, the lens 3 and the image sensor 2 a in theimage capturing section 100. It should be noted that illustration of theother elements is omitted in FIG. 2. The lens 3 may be a lens unit thatis a group of lenses but is drawn in FIG. 2 as a single lens for thesake of simplicity. The light-transmitting plate 1 has twolight-transmitting areas 1L and 1R that have mutually different spectraltransmittance characteristics, and transmits the incoming light at leastpartially. The lens 3 is a known lens and condenses the light that hasbeen transmitted through the light-transmitting plate 1, thereby imagingthe light on the imaging area 2 b of the image sensor 2 a. It should benoted that the relative arrangement of the respective members shown inFIG. 2 is only an example and does not always have to be adoptedaccording to the present invention. For example, as long as the lens 3can produce an image on the imaging area 2 b, the lens 3 may be arrangedmore distant from the image sensor 2 a than the light-transmitting plate1 is. Optionally, the lens 3 and the light-transmitting plate 1 may becombined together.

FIG. 3 is a front view of the light-transmitting plate 1 of thisembodiment. Even though the light-transmitting plate 1 of thisembodiment, as well as the lens 3, has a circular shape, thelight-transmitting plate 1 may also have any other shape. Thelight-transmitting plate 1 is split into two light-transmitting areas 1Land 1R, which are respectively located on the left- and right-hand sideson the paper. In this embodiment, the area 1L is implemented as a cyan(Cy) color filter (which will be referred to herein as a “Cy filter”),and the area 1R is implemented as a yellow (Ye) color filter (which willbe referred to herein as a “Ye filter”). Nevertheless, unlike ordinaryCy and Ye filters, the Cy and Ye filters of this embodiment are designedso that their transmittance with respect to light including the colorgreen (G) components becomes low in either the longer wavelength rangeor the shorter wavelength range. That is to say, the Cy filter isdesigned to mainly transmit a light ray falling within the color blue(B) wavelength range and a light ray falling within the shorter part(G1) of the color green (G) wavelength range. On the other hand, the Yefilter is designed to mainly transmit a light ray falling within thelonger part (G2) of the color green (G) wavelength range and a light rayfalling within the color red (R) wavelength range.

FIG. 4 shows exemplary spectral transmittance characteristics of thelight-transmitting areas 1R and 1L. In FIG. 4, the abscissa representsthe wavelength λ of the incoming light and the ordinate represents theoptical transmittance. In this example, the range of approximately400-500 nm is defined to be the B wavelength range, the range ofapproximately 500-600 nm is defined to be the G wavelength range, andthe range of approximately 600-700 nm is defined to be the R wavelengthrange. Also, in the G wavelength range, the range of approximately500-550 nm is defined to be the G1 wavelength range and the range ofapproximately 550-600 nm is defined to be the G2 wavelength range.However, these definitions are made just for convenience sake. And itmay be determined arbitrarily what wavelength range represents whatcolor. As shown in FIG. 4, both of the Cy and Ye filters of thisembodiment transmit only a half of the light with G components comparedto ordinary Cy and Ye filters. As a result, the sum of the functionrepresenting the spectral transmittance characteristic of the Cy filterand the function representing the spectral transmittance characteristicof the Ye filter becomes a function representing the spectraltransmittance characteristic of transparent light. Consequently, it ispossible to prevent the light-transmitting plate 1 from coloring theincoming light.

FIG. 5 illustrates only some of the photosensitive cells 60 that arearranged in columns and rows on the imaging area 2 b of the image sensor2 a. Each of those photosensitive cells 60 typically includes aphotodiode and performs photoelectric conversion and outputs aphotoelectrically converted signal (pixel signal) representing thequantity of light received. A color filter (which will also be referredto herein as a “color element”) is arranged closer to the light sourceso as to face each of those photosensitive cells 60.

As shown in FIG. 5, the color filter arrangement of this embodiment ismade up of multiple 2×2 unit matrices. In each unit matrix, a red (R)element is arranged at the row 1, column 1 position, a first green (G1)element is arranged at the row 1, column 2 position, a second green (G2)element is arranged at the row 2, column 1 position, and a blue (B)element is arranged at the row 2, column 2 position. This is a so-calledBayer arrangement but the two green elements thereof have a differentspectral transmittance characteristic from an ordinary green element. Inthe following description of this embodiment, the B element and aphotosensitive cell that faces it will be sometimes referred to hereinas a “first pixel”. Likewise, the G1 element and a photosensitive cellthat faces it will be sometimes referred to herein as a “second pixel”,the G2 element and a photosensitive cell that faces it will be sometimesreferred to herein as a “third pixel”, and the R element and aphotosensitive cell that faces it will be sometimes referred to hereinas a “fourth pixel”. As shown in FIG. 5, this image sensor 2 a has aconfiguration in which a plurality of pixel blocks 40, each consistingof the first through fourth pixels, are arranged two-dimensionally onthe imaging area 2 b.

FIG. 6 shows the spectral transmittance characteristics of therespective color elements of this embodiment. In FIG. 6, the one-dotchain curve represents the transmittance of the R element, the solidcurves represent the spectral transmittance characteristics of the G1and G2 elements, and the two-dot chain curve represents the spectraltransmittance characteristic of the B element. This embodiment ischaracterized by the spectral transmittance characteristics of the G1and G2 elements. Specifically, the peak of transmittance of the G1element has shifted toward the color blue range (i.e., toward theshorter wavelength range), while the peak of transmittance of the G2element has shifted toward the color red range (i.e., toward the longerwavelength range). Also, the sum of the respective transmittances of theB and G1 elements becomes substantially the same as the spectraltransmittance characteristic of the light-transmitting area 1L of thelight-transmitting plate 1. Likewise, the sum of the respectivetransmittances of the R and G2 elements becomes substantially the sameas the spectral transmittance characteristic of the light-transmittingarea 1R of the light-transmitting plate 1.

With such a configuration adopted, the first through fourth pixels ofthis embodiment mainly sense light rays falling within the R, G1, G2 andB wavelength ranges, respectively, and output photoelectricallyconverted signals representing the respective intensities of light raysfalling within those wavelength ranges.

The light-transmitting areas 1L and 1R of the light-transmitting plate 1and the respective elements of the image sensor 2 a may be made of aknown pigment or a multilayer dielectric film, for example. Ideally, theCy filter of the light-transmitting plate 1 should be designed totransmit only B and G1 light rays and the Ye filter thereof should bedesigned to transmit only R and G2 light rays. Actually, however, thosefilters may transmit some light rays representing other colors as wellas shown in FIG. 4. Likewise, it is ideal to design R, B, G1 and G2elements that transmit only R, B, G1 and G2 rays, respectively. However,those elements may actually transmit some light rays representing othercolors as well as shown in FIG. 6.

With such a configuration adopted, the light incident on this imagecapture device during an exposure process is transmitted through thelight-transmitting plate 1, the lens 3 and the infrared cut filter 4,imaged on the imaging area 2 b of the image sensor 2 a, and thenphotoelectrically converted by respective photosensitive cells 60. Then,photoelectrically converted signals are output from the photosensitivecells 60 to the signal processing section 200 by way of the signalgenerating and receiving section 5. In the signal processing section200, the image processing section 7 generates two multi-viewpoint imagesbased on the signals received.

FIG. 7 schematically illustrates the states of the light-transmittingplate 1 while images are going to be captured according to thisembodiment. The image capture device of this embodiment captures imagestwice by rotating the light-transmitting plate 1 as shown in portions(a) and (b) of FIG. 7 and generates a pair of multi-viewpoint imagesthrough arithmetic processing. Specifically, as disclosed in Non-PatentDocument No. 1, the light-transmitting plate 1 may be rotated by puttinga belt on the light-transmitting plate 1 and by running the belt with amotor. The rotating and driving section 9 rotates the light-transmittingplate 1 using such a mechanism and the image sensor 2 a obtains pixelsignals in the states shown in portions (a) and (b) of FIG. 7.

As can be seen, the Cy and Ye filters change their positions with eachother before and after the rotation. In this description, thelight-transmitting area shown on the left-hand side on the paper will beidentified herein by 1L and the light-transmitting area shown on theright-hand side on the paper by 1R. That is why in the state shown inportion (b) of FIG. 7 in which the light-transmitting plate 1 in thestate shown in portion (a) of FIG. 7 has rotated 180 degrees, the Yefilter is located in the light-transmitting area 1L and the Cy filter islocated in the light-transmitting area 1R.

Hereinafter, it will be described how this image capture device operateswhen an image of the subject is captured through the light-transmittingplate 1. As for the respective pixel signals of the image sensor 2 a,signals representing the respective intensities of light rays that havebeen transmitted through the R, G1, G2 and B elements andphotoelectrically converted will be identified herein by Rs, G1 s, G2 s,and Bs, respectively. If image capturing sessions have been performed ntimes, the signals obtained as a result of the i^(th) image capturingsession (where i is an integer and 1≦i≦n) will be identified herein byRs(i), G1 s(i), G2 s(i) and Bs(i), respectively. Even though n==2 inthis embodiment, n≧3 may be satisfied as well.

First of all, an image capturing session is carried out for the firsttime in the state shown in portion (a) of FIG. 7. In this case, R and G2rays are transmitted more through the light-transmitting area 1R of thelight-transmitting plate 1 than through the light-transmitting area 1L.That is why the pixel signals output from the photosensitive cells thatface the R and G2 elements of the image sensor 2 a include more signalcomponents representing the light transmitted through thelight-transmitting area 1R than signal components representing the lighttransmitted through the light-transmitting area 1L. B and G1 rays, onthe other hand, are transmitted more through the light-transmitting area1L of the light-transmitting plate 1 than through the light-transmittingarea 1R. That is why the pixel signals output from the photosensitivecells that face the B and G1 elements of the image sensor 2 a includemore signal components representing the light transmitted through thelight-transmitting area 1L than signal components representing the lighttransmitted through the light-transmitting area 1R. The pixel signalsobtained through this first image capturing session are sent from thesignal generating and receiving section 5 to the image processingsection 7, where image signals are generated and retained.

Next, the light-transmitting plate 1 in the state shown in portion (a)of FIG. 7 is rotated 180 degrees on its center as the axis of rotationby the rotating and driving section 9 to enter the state shown inportion (b) of FIG. 7. In this state, an image capturing session iscarried out for the second time, and the light-transmitting plate 1assumes the opposite state from the one described above. Thus, the pixelsignals output from the photosensitive cells that face the R and G2elements of the image sensor 2 a include more signal componentsrepresenting the light transmitted through the light-transmitting area1L than signal components representing the light transmitted through thelight-transmitting area 1R. On the other hand, the pixel signals outputfrom the photosensitive cells that face the B and G1 elements of theimage sensor 2 a include more signal components representing the lighttransmitted through the light-transmitting area 1R than signalcomponents representing the light transmitted through thelight-transmitting area 1L. In this case, the pixel signals are alsosent from the signal generating and receiving section 5 to the imageprocessing section 7, where image signals are generated and retained.

Based on the pixel signals Rs(1), Rs(2), G1 s(1), G1 s(2), G2 s(1), G2s(2), Bs(1) and Bs(2) that have been obtained as a result of these twoimage capturing sessions, signal arithmetic operations are performed,thereby generating multi-viewpoint images. In this case, signalsrepresenting the intensities of light rays to be transmitted through thelight-transmitting areas 1L and 1R and then photoelectrically convertedin a situation where the light-transmitting areas 1L and 1R and therespective color elements on the image sensor 2 a are supposed to becompletely transparent are identified herein by L and R, respectively.In that case, the relations between the pixel signals Rs(1), Rs(2), G1s(1), G1 s(2), G2 s(1), G2 s(2), Bs(1) and Bs(2) and the signals L and Rare given by the following Equations (1) through (4), in which the R,G1, G2 and B components of the signals L and R are respectivelyidentified by the subscripts r, g1, g2 and b attached to the signs L andR. Then, the R, G1, G2 and B components are represented by the followingEquations (1), (2), (3) and (4), respectively:

$\begin{matrix}{\begin{pmatrix}{{Rs}(1)} \\{{Rs}(2)}\end{pmatrix} = {\begin{pmatrix}{{rM}\; 11} & {{rM}\; 12} \\{{rM}\; 21} & {{rM}\; 22}\end{pmatrix}\begin{pmatrix}L_{r} \\R_{r}\end{pmatrix}}} & (1) \\{\begin{pmatrix}{G\; 1{s(1)}} \\{G\; 1{s(2)}}\end{pmatrix} = {\begin{pmatrix}{g\; 1M\; 11} & {g\; 1M\; 12} \\{g\; 1M\; 21} & {g\; 1M\; 22}\end{pmatrix}\begin{pmatrix}L_{g\; 1} \\R_{g\; 1}\end{pmatrix}}} & (2) \\{\begin{pmatrix}{G\; 2{s(1)}} \\{G\; 2{s(2)}}\end{pmatrix} = {\begin{pmatrix}{g\; 2M\; 11} & {g\; 2M\; 12} \\{g\; 2M\; 21} & {g\; 2M\; 22}\end{pmatrix}\begin{pmatrix}L_{g\; 2} \\R_{g\; 2}\end{pmatrix}}} & (3) \\{\begin{pmatrix}{{Bs}(1)} \\{{Bs}(2)}\end{pmatrix} = {\begin{pmatrix}{{bM}\; 11} & {{bM}\; 12} \\{{bM}\; 21} & {{bM}\; 22}\end{pmatrix}\begin{pmatrix}L_{b} \\R_{b}\end{pmatrix}}} & (4)\end{matrix}$

In these Equations (1) through (4), each of the 2×2 matrix elements onthe right side is a factor of proportionality represented by thewavelength integral value of the transmittance of a light ray that hasbeen transmitted through each light-transmitting area of thelight-transmitting plate 1 and incident on photosensitive cells facingthe respective color elements of the image sensor 2 a. For example, rM11and rM12 of Equation (1) are factors about the first image capturingsession and are represented by the following Equations (5) and (6),respectively:rM11=K∫ _(r) Cy(λ)O(λ)R(λ)dλ  (5)rM12=K∫ _(r) Ye(λ)O(λ)R(λ)dλ  (6)

In Equations (5) and (6), K is a constant of proportionality. Andsupposing the wavelength of the incident light is λ, the spectraltransmittance characteristics of the Cy and Ye filters of thelight-transmitting plate 1 are identified by Cy(λ) and Ye(λ),respectively, the spectral transmittance characteristic of the R elementof the image sensor 2 a is identified by R(λ), and the spectraltransmittance characteristic of every other element including the lens3, the infrared cut filter 4 and the image sensor 2 a itself isidentified by O(λ). The sign r under the integral sign indicates thatthe integration operation is performed on the R wavelength range. Forexample, if the wavelength ranges are defined as described above, theintegration operation is performed in the range in which λ=600 to 700nm.

Also, in Equation (1), rM21 and rM22 are factors for the second imagecapturing session and rM21==mM12 and rM22==mM11. Since the two imagecapturing sessions are supposed to be performed continuously within ashort period of time according to this embodiment, a variation in thequantity of the incoming light that could be caused between the twoimage capturing sessions is not taken into account. Thus, the twoequations described above are satisfied.

Likewise, the 2×2 matrix elements on the right side of Equations (2)through (4) can also be obtained through similar calculations just byreplacing R(λ) of Equations (5) and (6) with the spectral transmittancecharacteristic of each color element and by changing the interval ofintegration into the wavelength range of that color. That is to say,g1M11, g1M12, g2M11, g2M12, bM11 and bM12 are given by the followingEquations (7) through (12), respectively. In the following equations,the spectral transmittance characteristics of the G1, G2, and B elementsare identified by G1(λ), G2(λ) and B(λ), respectively:g1M11=K∫ _(g1) Cy(λ)O(λ)G1(λ)dλ  (7)g1M12=K∫ _(g1) Ye(λ)O(λ)G1(λ)dλ  (8)g2M11=K∫ _(g2) Cy(λ)O(λ)G2(λ)dλ  (9)g2M12=K∫ _(g2) Ye(λ)O(λ)G2(λ)dλ  (10)bM11=K∫ _(b) Cy(λ)O(λ)B(λ)dλ  (11)bM12=K∫ _(b) Ye(λ)O(λ)B(λ)dλ  (12)

In these Equations (7) through (12), the signs g1, g2 and b under theintegral sign indicate that the integration operation is performed onthe wavelength ranges of G1, G2 and B, respectively. For example, if thewavelength ranges defined above are adopted, the integration operationis performed in the range in which λ==500 to 550 nm for Equations (7)and (8), in the range in which λ=550 to 600 nm for Equations (9) and(10), and in the range in which λ==400 to 500 nm for Equations (11) and(12), respectively.

As for the factors for this second image capturing session, g1M21=g1M12,g1M22=g1M11, g2M21=g2M12, g2M22=g2M11, bM21=bM12, and bM22=bM11 aresatisfied just like the R component.

In the foregoing description, the integration operation is supposed tobe performed only within the wavelength range of each particular colorcomponent to obtain the respective matrix elements. However, such amethod is not necessarily adopted. Alternatively, by using differentfunctions O(λ) representing the optical transmittance of a componentother than the color filters for the B, G1, G2 and R color components,the integration operation may be performed over the entire wavelengthrange of visible radiation (e.g., in the range of 400 nm to 700 nm).Still alternatively, the respective matrix elements may also be obtainedby performing an integration operation on the entire wavelength range ofvisible radiation with the function O(λ) excluded and by multiplying theresult of the integration operation by a constant to be determined withthe effect of O(λ) taken into account.

The image processing section 7 performs the processing of multiplyingboth sides of each of Equations (1) to (4) by the inverse matrix of the2×2 matrix of that equation from the left to the right, therebyobtaining the respective color components of the signal L representingan image to be produced by the light that has been transmitted throughthe left-side area 1L of the light-transmitting plate 1 and therespective color components of the signal R representing an image to beproduced by the light that has been transmitted through the right-sidearea 1R thereof. The respective color components Lr, Lg1, Lg2 and Lb ofthe signal L and the respective color components Rr, Rg1, Rg2 and Rb ofthe signal R are calculated by the following Equations (13) through(16):

$\begin{matrix}{\begin{pmatrix}L_{r} \\R_{r}\end{pmatrix} = {\begin{pmatrix}{{rM}\; 11} & {{rM}\; 12} \\{{rM}\; 21} & {{rM}\; 22}\end{pmatrix}^{- 1}\begin{pmatrix}{{Rs}(1)} \\{{Rs}(2)}\end{pmatrix}}} & (13) \\{\begin{pmatrix}L_{g\; 1} \\R_{g\; 1}\end{pmatrix} = {\begin{pmatrix}{g\; 1M\; 11} & {g\; 1M\; 12} \\{g\; 1M\; 21} & {g\; 1M\; 22}\end{pmatrix}^{- 1}\begin{pmatrix}{G\; 1{s(1)}} \\{G\; 1{s(2)}}\end{pmatrix}}} & (14) \\{\begin{pmatrix}L_{g\; 2} \\R_{g\; 2}\end{pmatrix} = {\begin{pmatrix}{g\; 2M\; 11} & {g\; 2M\; 12} \\{g\; 2M\; 21} & {g\; 2M\; 22}\end{pmatrix}^{- 1}\begin{pmatrix}{G\; 2{s(1)}} \\{G\; 2{s(2)}}\end{pmatrix}}} & (15) \\{\begin{pmatrix}L_{b} \\R_{b}\end{pmatrix} = {\begin{pmatrix}{{bM}\; 11} & {{bM}\; 12} \\{{bM}\; 21} & {{bM}\; 22}\end{pmatrix}^{- 1}\begin{pmatrix}{{Bs}(1)} \\{{Bs}(2)}\end{pmatrix}}} & (16)\end{matrix}$

Furthermore, by making the calculations represented by the followingEquations (17) and (18), the image processing section 7 generatessignals L and R that form those light intensity images.L=L _(r) +L _(g1) +L _(g2) +L _(b)  (17)R=R _(r) +R _(g1) +R _(g2) +R _(b)  (18)

The image processing section 7 performs such signal arithmeticprocessing on each of the pixel blocks 40 shown in FIG. 5. A set of thesignals L, R that have been generated on a pixel block (40) basis inthis manner is output as a pair of multi-viewpoint images from the imageinterface section 8.

As described above, according to this embodiment, by using thelight-transmitting plate 1 in which Cy and Ye filters are arranged sideby side in the horizontal direction (x direction) and the image sensor 2a with the Bayer arrangement in which two kinds of G elements withmutually different transmittances are arranged, image capturing sessionsare performed twice before and after the light-transmitting plate 1 isrotated 180 degrees. By performing arithmetic processing on each colorsignal using the 2×2 matrix, multi-viewpoint images can be obtained.Since the light-transmitting plate 1 is designed to transmitcomplementary color rays (Cy, Ye), color multi-viewpoint images can beobtained with good sensitivity by using incoming light more efficientlythan in the related art.

In the embodiment described above, the Cy and Ye filters of thelight-transmitting plate 1 have different spectral transmittancecharacteristics as shown in FIG. 4 but are designed so that the integralvalues of their transmittances over the entire wavelength range ofvisible radiation are substantially equal to each other. However, the Cyand Ye filters do not have to be designed in this manner but theintegral value of the transmittances of one of these two filters may belarger than that of the other. For example, as for the transmittance ofa green ray, the Cy and Ye filters may also be designed to respectivelytransmit approximately 70% and approximately 30% of the green ray asshown in FIG. 8A. No matter what spectral transmittance characteristicsthe Cy and Ye filters have, multi-viewpoint images can also be generatedby performing the signal processing described above as long as theirfunction forms are known.

Also, in the embodiment described above, Cy and Ye filters are supposedto be used in the light-transmitting areas 1L and 1R of thelight-transmitting plate 1. However, any other combination of colorfilters may also be used. In any case, if the sum of the spectraltransmittance characteristics of those color filters can be called asubstantially transparent characteristic, the effects of the embodimentdescribed above can also be achieved. In this description, if somethingis “transparent”, then it means that it has a characteristic with atransmittance of approximately 80% or more with respect to a light rayfalling within an arbitrary part of the visible radiation wavelengthrange. Also, in one embodiment, the two light-transmitting areas 1L and1R may be configured so that the sum of functions representing theirspectral transmittance characteristics does not have wavelengthdependence. In this description, if something “does not have wavelengthdependence”, then it means that the ratio of the minimum value to themaximum value of a function representing the spectral transmittancecharacteristic in the visible radiation wavelength range falls withinthe range of approximately 0.8 to 1.0.

Furthermore, the two light-transmitting areas 1L and 1R of thelight-transmitting plate 1 do not have to be arranged to form two halvesof the light-transmitting plate 1. Alternatively, the light-transmittingarea 1L may form part of the left half of the light-transmitting plate1, the light-transmitting area 1R may form part of the right half of thelight-transmitting plate 1 and the rest of the light-transmitting plate1 may be an opaque member as shown in FIG. 8B. If such an arrangement isadopted, the quantity of the incoming light that can be used willdecrease, but more perceptible parallax information can be obtained,compared to the arrangement shown in FIG. 3. Even if such an arrangementis adopted, the light-transmitting areas 1L and 1R can still be arrangedsymmetrically with respect to the center of the light-transmitting plate1. Also, these two light-transmitting areas 1L and 1R do not have to bedesigned to have the same size. Even if their sizes are different fromeach other, their difference can also be compensated for by signalprocessing.

Also, as for the color arrangement of the image sensor 2 a, even thougha Bayer type arrangement is supposed to be used in the embodimentdescribed above, such an arrangement does not have to be used. Forexample, there is no problem even if the B and G2 elements on the secondrow change their positions with each other as shown in FIG. 8C.Furthermore, the pixel arrangement of the image sensor 2 a does not haveto a tetragonal arrangement such as the one shown in FIG. 5 but may alsobe an oblique arrangement in which pixels are arranged obliquely to thex- and y-axes.

Furthermore, as for the color elements of the image sensor 2 a, R and Belements are supposed to be used besides the first and second Gelements. However, this is only an example. Alternatively, there is noproblem even if the R element is replaced with a magenta element withhigh R ray transmittance and if the B element is replaced with a magentaelement with high B ray transmittance. Rather, if those elements areused, the incoming light can be used even more efficiently, which isadvantageous. Furthermore, the G1 and G2 elements may have mutuallydifferent integral values of transmittances over the entire wavelengthrange of visible radiation. For example, as shown in FIG. 8D, thetransmittances of the G1 and G2 elements with respect to a green ray maybe set to be approximately 70% and approximately 30%, respectively. Evenif the spectral transmittance characteristics of the respective elementsare different from those of the embodiment described above but if theirfunction forms are known, the signal processing can also be carried outjust as described above.

Furthermore, in the embodiment described above, two kinds of filters G1and G2 are supposed to be used as green elements in each pixel block ofthe image sensor 2 a. However, they may be replaced with ordinary greenelements which transmit most of the green ray. In that case, thecombination of color elements used will be quite the same as that of anormal Bayer arrangement and Equations (2) and (3) will be combined intoa single equation.

Furthermore, in the embodiment described above, the positions of colorfilters arranged over the two light-transmitting areas are supposed tobe changed with each other by getting the light-transmitting plate 1rotated by the rotating and driving section 9. However, those colorfilters may also change their positions even if the light-transmittingplate 1 is not rotated. For example, the positions of those filters mayalso be changed by sliding the color filters in one direction as shownin FIG. 9. A sliding plate 1 a in which three color filters are arrangedis attached to the light-transmitting plate 1 shown in FIG. 9. And bysliding the sliding plate 1 a, the light-transmitting areas 1L and 1Rcan change their spectral transmittance characteristics with each other.In the example illustrated in FIG. 9, a Cy filter is arranged at thecenter of the sliding plate 1 a and Ye filters are arranged at bothends. Without the sliding plate 1 a, the light-transmitting areas 1L and1R are both transparent as shown in FIG. 9( a). An image capturingsession is performed for the first time with the Cy and Ye filtersarranged over the light-transmitting areas 1L and 1R, respectively, asshown in FIG. 9( b). Subsequently, an image capturing session isperformed for the second time with the Ye and Cy filters arranged overthe light-transmitting areas 1L and 1R, respectively, as shown in FIG.9( c). Even with such an arrangement adopted, the effects of theembodiment described above can also be achieved.

Embodiment 2

Hereinafter, a second embodiment of the present invention will bedescribed. Major differences between this and first embodiments lie inthe configuration of the light-transmitting plate 1 and in theprocessing to be performed by the image processing section 7. Thefollowing description of this second embodiment will be focused on thosedifferences from the first embodiment and their common features will notbe described all over again.

FIG. 10 is a front view of a light-transmitting plate 1 according tothis embodiment. This light-transmitting plate 1 has a circular shapeand has light-transmitting areas 1L, 1R and two transparent areas W. Thelight-transmitting areas 1L and 1R are arranged symmetrically withrespect to the y-axis, while the two transparent areas W are arrangedsymmetrically with respect to the x-axis. The light-transmitting areas1L, 1R and the two transparent areas W each have a fan shape. As in thefirst embodiment, Cy and Ye filters are arranged in thelight-transmitting areas 1L and 1R, respectively. Those filters may havethe same characteristics as their counterparts of the first embodiment.The transparent areas W may be made of a material (such as glass or air)which transmits most of incoming light (visible radiation) irrespectiveof the wavelength. To use incoming light as efficiently as possible, theoptical transmittance of the transparent areas W may be set to be a highvalue. And the light-transmitting areas 1L and 1R are designed to havean equal area.

The members other than the light-transmitting plate 1 of this embodimenthave the same configurations as their counterparts of the firstembodiment. And the operation of the device according to this embodimentis also controlled through two image capturing sessions as in the firstembodiment described above. But the image processing section 4 of thisembodiment performs different arithmetic processing from the firstembodiment.

Hereinafter, the flow of the arithmetic processing of this embodimentwill be described. First of all, the subject image to be produced by thelight rays that have been transmitted through the light-transmittingareas 1L and 1R and transparent areas W of the light-transmitting plate1 is represented by two different expressions. Specifically, the subjectimage is represented by a first expression in which an image signal isexpressed by R, G1 and B components, and is also represented by a secondexpression in which an image signal is expressed by R, G2 and Bcomponents. Next, as for each of these two expressions, images producedby the light rays that have been incident on the respective areas of thelight-transmitting plate 1 are calculated using signals representing therespective color components. Finally, the signals representing thosecolor components are synthesized together, thereby generating colormulti-viewpoint images. The relation between the signals representingthe light rays that have actually been incident on the respective areasand the signals representing those color components can be expressed byan equation that uses a 3×3 matrix. In this case, signals representingthe images to be produced by the light rays that have been transmittedthrough the light-transmitting areas 1L and 1R and the transparent areasW in a situation where the areas 1L, 1R and W of the light-transmittingplate 1 and the respective color elements on the image sensor 2 a areall supposed to be completely transparent are identified herein by L, Rand C, respectively. Also, as for those images, signals associated withthe i^(th) (where i=1, 2) image capturing session are identified by thesubscript i in parentheses added to their sign. Then, the relationsbetween the pixel signals Rs(1), Rs(2), G1 s(1), G1 s(2), G2 s(1), G2s(2), Bs(1) and Bs(2) and the signals L, R and C are given by thefollowing Equations (19) and (20) for the first image capturing sessionand by the following Equations (21) and (22) for the second imagecapturing session, respectively:

$\begin{matrix}{\begin{pmatrix}{{Rs}(1)} \\{G\; 1{s(1)}} \\{{Bs}(1)}\end{pmatrix} = {\begin{pmatrix}{{Mu}\; 11} & {{Mu}\; 12} & {{Mu}\; 13} \\{{Mu}\; 21} & {{Mu}\; 22} & {{Mu}\; 23} \\{{Mu}\; 31} & {{Mu}\; 32} & {{Mu}\; 33}\end{pmatrix}\begin{pmatrix}L_{1} \\R_{1} \\C_{1}\end{pmatrix}}} & (19) \\{\begin{pmatrix}{{Rs}(1)} \\{G\; 2{s(1)}} \\{{Bs}(1)}\end{pmatrix} = {\begin{pmatrix}{{Mu}\; 11} & {{Mu}\; 12} & {{Mu}\; 13} \\{{Mu}\; 21^{\prime}} & {{Mu}\; 22^{\prime}} & {{Mu}\; 23^{\prime}} \\{{Mu}\; 31} & {{Mu}\; 32} & {{Mu}\; 33}\end{pmatrix}\begin{pmatrix}L_{1} \\R_{1} \\C_{1}\end{pmatrix}}} & (20) \\{\begin{pmatrix}{{Rs}(2)} \\{G\; 1{s(2)}} \\{{Bs}(2)}\end{pmatrix} = {\begin{pmatrix}{{Mv}\; 11} & {{Mv}\; 12} & {{Mv}\; 13} \\{{Mv}\; 21} & {{Mv}\; 22} & {{Mv}\; 23} \\{{Mv}\; 31} & {{Mv}\; 32} & {{Mv}\; 33}\end{pmatrix}\begin{pmatrix}L_{2} \\R_{2} \\C_{2}\end{pmatrix}}} & (21) \\{\begin{pmatrix}{{Rs}(2)} \\{G\; 2{s(2)}} \\{{Bs}(2)}\end{pmatrix} = {\begin{pmatrix}{{Mv}\; 11} & {{Mv}\; 12} & {{Mv}\; 13} \\{{Mv}\; 21^{\prime}} & {{Mv}\; 22^{\prime}} & {{Mv}\; 23^{\prime}} \\{{Mv}\; 31} & {{Mv}\; 32} & {{Mv}\; 33}\end{pmatrix}\begin{pmatrix}L_{2} \\R_{2} \\C_{2}\end{pmatrix}}} & (22)\end{matrix}$

In these Equations (19) through (22), each of the 3 ×3 matrix elementsis a factor of proportionality represented by the wavelength integralvalue of the transmittance of a light ray that has been transmittedthrough each area of the light-transmitting plate 1 and incident onphotosensitive cells facing the respective color elements of the imagesensor 2 a. For example, the respective elements of the 3×3 matrix ofEquation (19) can be calculated by the following Equations (23) through(31), in which K′ is a factor of proportionality and W(λ) is functionrepresenting the spectral transmittance characteristic of the W areas ofthe light-transmitting plate 1.Mu11=K′∫ _(r) Cy(λ)O(λ)R(λ)dλ  (23)Mu12=K′∫ _(r) Ye(λ)O(λ)R(λ)dλ  (24)Mu13=K′∫ _(r) W(λ)O(λ)R(λ)dλ  (25)Mu21=K′∫ _(g1) Cy(λ)O(λ)G1(λ)dλ  (26)Mu22=K′∫ _(g1) Ye(λ)O(λ)G1(λ)dλ  (27)Mu23=K′∫ _(g1) W(λ)O(λ)G1(λ)dλ  (28)Mu31=K′∫ _(b) Cy(λ)O(λ)B(λ)dλ  (29)Mu32=K′∫ _(b) Ye(λ)O(λ)B(λ)dλ  (30)Mu33=K′∫ _(b) W(λ)O(λ)B(λ)dλ  (31)

In Equations (19) and (20), only parameters concerning the G1 and G2components are different and parameters concerning the other colorcomponents are the same. The same can be said about Equations (21) and(22). If each of Equations (19) through (22) is multiplied on both sidesby the inverse matrix of the 3×3 matrix on the right side from the leftto the right, images represented by the light rays that have beentransmitted through the respective areas can be obtained as given by thefollowing Equations (32) through (35), in which the 3×3 matrix on theright side is the inverse matrix of the 3×3 matrix in Equations (19) to(22).

$\begin{matrix}{\begin{pmatrix}L_{1} \\R_{1} \\C_{1}\end{pmatrix} = {\begin{pmatrix}{{iMu}\; 11} & {{iMu}\; 12} & {{iMu}\; 13} \\{{iMu}\; 21} & {{iMu}\; 22} & {{iMu}\; 23} \\{{iMu}\; 31} & {{iMu}\; 32} & {{iMu}\; 33}\end{pmatrix}\begin{pmatrix}{{Rs}(1)} \\{G\; 1{s(1)}} \\{{Bs}(1)}\end{pmatrix}}} & (32) \\{\begin{pmatrix}L_{1} \\R_{1} \\C_{1}\end{pmatrix} = {\begin{pmatrix}{{iMu}\; 11^{\prime}} & {{iMu}\; 12^{\prime}} & {{iMu}\; 13^{\prime}} \\{{iMu}\; 21^{\prime}} & {{iMu}\; 22^{\prime}} & {{iMu}\; 23^{\prime}} \\{{iMu}\; 31^{\prime}} & {{iMu}\; 32^{\prime}} & {{iMu}\; 33^{\prime}}\end{pmatrix}\begin{pmatrix}{{Rs}(1)} \\{G\; 2{s(1)}} \\{{Bs}(1)}\end{pmatrix}}} & (33) \\{\begin{pmatrix}L_{2} \\R_{2} \\C_{2}\end{pmatrix} = {\begin{pmatrix}{{iMv}\; 11} & {{iMv}\; 12} & {{iMv}\; 13} \\{{iMv}\; 21} & {{iMv}\; 22} & {{iMv}\; 23} \\{{iMv}\; 31} & {{iMv}\; 32} & {{iMv}\; 33}\end{pmatrix}\begin{pmatrix}{{Rs}(2)} \\{G\; 1{s(2)}} \\{{Bs}(2)}\end{pmatrix}}} & (34) \\{\begin{pmatrix}L_{2} \\R_{2} \\C_{2}\end{pmatrix} = {\begin{pmatrix}{{iMv}\; 11^{\prime}} & {{iMv}\; 12^{\prime}} & {{iMv}\; 13^{\prime}} \\{{iMv}\; 21^{\prime}} & {{iMv}\; 22^{\prime}} & {{iMv}\; 23^{\prime}} \\{{iMv}\; 31^{\prime}} & {{iMv}\; 32^{\prime}} & {{iMv}\; 33^{\prime}}\end{pmatrix}\begin{pmatrix}{{Rs}(2)} \\{G\; 2{s(2)}} \\{{Bs}(2)}\end{pmatrix}}} & (35)\end{matrix}$

The image processing section 7 generates multi-viewpoint images byperforming arithmetic processing on the two sets of L1, R1, C1, L2, R2and C2 that have been calculated by Equations (32) through (35).Specifically, luminance signals L, R and C that will form themulti-viewpoint images may be generated in the following manner, forexample. First of all, the average of L1 of Equation (32), L1 ofEquation (33), L2 of Equation (34), and L2 of Equation (34) iscalculated as the luminance signal of L. In the same way, as for R andC, the average of the respective signals is calculated as the luminancesignal:

-   -   Luminance of L: {(L1 of Equation (32)+(L1 of Equation (33)+(L2        of Equation (34)+(L2 of Equation (35)}/4    -   Luminance of R: {(R1 of Equation (32)+(R1 of Equation (33)+(R2        of Equation (34)+(R2 of Equation (35)}/4    -   Luminance of C: {(C1 of Equation (32)+(C1 of Equation (33)+(C2        of Equation (34)+(C2 of Equation (35)}/4

Next, as for the color signals, the same signals are used for each of L,R and C. The R, G and B values may be obtained in the following manner,for example:

-   -   R value: (Rs(1)+Rs(2))/2    -   G value: (G1 s(1)+G2 s(1)+G1 s(2)+G2 s(2))/4    -   B value: (Bs(1)+Bs(2))/2

Next, color difference signals are generated based on these R, G and Bvalues. In this case, the following color difference signals R-Y and B-Yare calculated by the NTSC method:

-   -   R−Y=0.7R−0.59G−0.11B    -   B−Y=−0.3R−0.59G+0.89B

Finally, using the luminance signal of L, R and C as YL, R, G and Bvalues are calculated in the following manner for each of L, R and C:R=(R−Y)+YLB=(B−Y)+YLG=(YL−0.3R−0.11B)/0.59

By performing such arithmetic processing, color multi-viewpoint imagescan be generated. According to this method, the luminance information ofmulti-viewpoint images is generated primarily with color informationsuperposed additionally. This approach is taken because human eyes aresensitive to luminance information but less sensitive to colorinformation by nature.

As described above, according to this embodiment, a light-transmittingplate 1 in which Cy and Ye filters are arranged horizontally (i.e., inthe x direction) with transparent areas W added and a color image sensorwith the Bayer arrangement which uses two kinds of G elements withmutually different transmittances are used. By performing imagecapturing sessions twice before and after the light-transmitting plate 1is rotated 180 degrees and by performing arithmetic processing on therespective color signals using the 3×3 matrix, multi-viewpoint imagescan be obtained. Since the light-transmitting plate 1 is made up ofcomplementary color filters and transparent members according to thisembodiment, multi-viewpoint images can be obtained with good sensitivityby using the incoming light highly efficiently.

In the embodiment described above, the areas of the Cy and Ye filtersand the transparent areas are supposed to have the same size in thelight-transmitting plate 1. However, this is only an example and thoseareas may have different sizes, too.

In the embodiments described above, the image processing is supposed tobe carried out by the image processing section 7 that is built in theimage capture device. However, such image processing may also be carriedout by another device that is provided independently of that imagecapture device. For example, even if a signal that has been obtained byan image capture device including the image capturing section 100 of theembodiment described above is loaded into another device (imageprocessor) to get a program defining the signal arithmetic processingdescribed above executed by a computer built in the image processor, theeffects of the embodiments described above can also be achieved.

INDUSTRIAL APPLICABILITY

A 3D image capture device according to an embodiment of the presentinvention can be used effectively in any camera that ever uses asolid-state image sensor. Examples of those cameras include consumerelectronic cameras such as digital cameras and digital camcorders andsolid-state surveillance cameras for industrial use.

REFERENCE SIGNS LIST

-   1 light-transmitting plate-   1L, 1R light-transmitting area-   2 solid-state image sensor-   2 a color solid-state image sensor-   3 optical lens-   4 infrared cut filter-   5 signal generating and image signal receiving section-   6 sensor driving section-   7 image processing section-   8 image interface section-   9 rotating and driving section-   10 pixel-   11 0-degree-polarization polarizer-   12 90-degree-polarization polarizer-   13 reflective mirror-   14 half mirror-   15 circular polarization filter-   16 driver that rotates polarization filter-   17, 18 polarization filter-   19 lens diaphragm-   20, 22, 23 light beam confining plate-   20 a color filter that transmits red-based light-   20 b color filter that transmits blue-based light-   21 photosensitive film-   22R, 23R R ray transmitting area of light beam confining plate-   22G, 23G G ray transmitting area of light beam confining plate-   22B, 23B B ray transmitting area of light beam confining plate-   29 image capture device-   30 memory-   40 pixel block-   60 photosensitive cell-   100 image capturing section-   200 signal processing section

The invention claimed is:
 1. A 3D image capture device comprising: alight-transmitting section including first and second light-transmittingareas, the first light-transmitting area having a property to transmitlight falling within a first wavelength range that is included in acolor blue wavelength range and light falling within a second wavelengthrange that is included in a color green wavelength range, the secondlight-transmitting area having a property to transmit light fallingwithin a third wavelength range, which is included in the color greenwavelength range and of which the wavelength is longer than the secondwavelength range, and light falling within a fourth wavelength rangethat is included in a color red wavelength range; an image sensor thatis arranged to receive the light that has been transmitted through thelight-transmitting section; a lens system that produces an image on animaging area of the image sensor; and an image capturing driver thatdrives the image sensor so as to perform image capturing sessions atleast twice in a row and that drives the light-transmitting section sothat the first and second light-transmitting areas change theirpositions with each other every time an image capturing session iscarried out, wherein the image sensor has a plurality of pixel blocksthat are arranged two-dimensionally on the imaging area, and whereineach of the plurality of pixel blocks includes first, second, third andfourth pixels that mainly sense light rays falling within the first,second, third and fourth wavelength ranges, respectively.
 2. The 3Dimage capture device of claim 1, further comprising an image processingprocessor that generates multi-viewpoint images based on a pixel signalsupplied from the image sensor.
 3. The 3D image capture device of claim1, wherein the first and second light-transmitting areas are configuredso that a sum of a a spectral transmittance characteristic of the firstlight-transmitting area represented by a function and a spectraltransmittance characteristic of the second light-transmitting arearepresented by another function does not have wavelength dependence. 4.The 3D image capture device of claim 1, wherein the light-transmittingsection further includes a transparent area.
 5. The 3D image capturedevice of claim 1, wherein the second wavelength range is from 500 nm to550 nm and the third wavelength range is from 550 nm to 600 nm.
 6. The3D image capture device of claim 1, wherein the image sensor has a Bayertype pixel arrangement.
 7. The 3D image capture device of claim 1,wherein the first and second light-transmitting areas are arrangedsymmetrically with respect to the center of the light-transmittingsection, and wherein the image capturing driver rotates thelight-transmitting section 180 degrees on the center of thelight-transmitting section as the axis of rotation, thereby changing thepositions of the first and second light-transmitting areas with eachother every image capturing session.
 8. An image processor thatgenerates multi-viewpoint images based on a signal supplied from a 3Dimage capture device, the device comprising: a light-transmittingsection including first and second light-transmitting areas, the firstlight-transmitting area having a property to transmit light fallingwithin a first wavelength range that is included in a color bluewavelength range and light falling within a second wavelength range thatis included in a color green wavelength range, the secondlight-transmitting area having a property to transmit light fallingwithin a third wavelength range, which is included in the color greenwavelength range and of which the wavelength is longer than the secondwavelength range, and light falling within a fourth wavelength rangethat is included in a color red wavelength range; an image sensor thatis arranged to receive the light that has been transmitted through thelight-transmitting section; a lens system that produces an image on animaging area of the image sensor; and an image capturing driver thatdrives the image sensor so as to perform image capturing sessions atleast twice in a row and that drives the light-transmitting section sothat the first and second light-transmitting areas change theirpositions with each other every time an image capturing session iscarried out, wherein the image sensor has a plurality of pixel blocksthat are arranged two-dimensionally on the imaging area, and whereineach of the plurality of pixel blocks includes first, second, third andfourth pixels that mainly sense light rays falling within the first,second, third and fourth wavelength ranges, respectively.